In this short consensus paper, we outline risks from upcoming, advanced AI systems. We examine large-scale social harms and malicious uses, as well as an irreversible loss of human control over autonomous AI systems. In light of rapid and continuing AI progress, we propose urgent priorities for AI R&D and governance.

Researchers in recent years have deployed reinforcement learning (RL) agents to solve increasingly challenging problems. As the trend continues, so has the development of new methods that enable the injection of "priors" (prior knowledge) into agents to help them better understand the structure of the world and come up with more effective solution strategies. In a new paper, researchers from Google, OpenAI, and DeepMind introduce "behaviour priors," a framework designed to capture common movement and interaction patterns that are shared across a set of related tasks or contexts. The researchers discuss how such behaviour patterns can be captured using probabilistic trajectory models and how they can be integrated effectively into RL schemes, such as for facilitating multi-task and transfer learning. Their method for learning behaviour priors can lead to significant speedups on complex tasks, the researchers say.

The conversation around Artificial Intelligence usually revolves around technology-focused topics: machine learning, conversational interfaces, autonomous agents, and other aspects of data science, math, and implementation. However, the history and evolution of AI is more than just a technology story. The story of AI is also inextricably linked with waves of innovation and research breakthroughs that run headfirst into economic and technology roadblocks. There seems to be a continuous pattern of discovery, innovation, interest, investment, cautious optimism, boundless enthusiasm, realization of limitations, technological roadblocks, withdrawal of interest, and retreat of AI research back to academic settings. These waves of advance and retreat seem to be as consistent as the back and forth of sea waves on the shore. This pattern of interest, investment, hype, then decline, and rinse-and-repeat is particularly vexing to technologists and investors because it doesn't follow the usual technology adoption lifecycle.

The conversation around Artificial Intelligence usually revolves around technology-focused topics: machine learning, conversational interfaces, autonomous agents, and other aspects of data science, math, and implementation. However, the history and evolution of AI is more than just a technology story. The story of AI is also inextricably linked with waves of innovation and research breakthroughs that run headfirst into economic and technology roadblocks. There seems to be a continuous pattern of discovery, innovation, interest, investment, cautious optimism, boundless enthusiasm, realization of limitations, technological roadblocks, withdrawal of interest, and retreat of AI research back to academic settings. These waves of advance and retreat seem to be as consistent as the back and forth of sea waves on the shore. This pattern of interest, investment, hype, then decline, and rinse-and-repeat is particularly vexing to technologists and investors because it doesn't follow the usual technology adoption lifecycle.

The conversation around Artificial Intelligence usually revolves around technology-focused topics: machine learning, conversational interfaces, autonomous agents, and other aspects of data science, math, and implementation. However, the history and evolution of AI is more than just a technology story. The story of AI is also inextricably linked with waves of innovation and research breakthroughs that run headfirst into economic and technology roadblocks. There seems to be a continuous pattern of discovery, innovation, interest, investment, cautious optimism, boundless enthusiasm, realization of limitations, technological roadblocks, withdrawal of interest, and retreat of AI research back to academic settings. These waves of advance and retreat seem to be as consistent as the back and forth of sea waves on the shore.

Until recently, a multinational company looking to help customers around the world book international travel would have had to build separate chatbots from scratch to converse in French, Hindi, Japanese or other languages. But thanks to artificial intelligence research breakthroughs that have enabled algorithms to more accurately parse nuances in the way different languages express concepts or structure sentences, it is now possible to build a single bot and use Microsoft Translator to translate questions and answers accurately enough for use in multiple countries. Over the past few years, Microsoft deep learning researchers were the first to achieve human parity milestones in developing algorithms that could perform about as well as a person on research benchmarks testing conversational speech recognition, reading comprehension, translation of news articles and other challenging language understanding tasks. Now, the benefits of those AI research breakthroughs are making their way into products from Azure to Bing. Search engineers are borrowing lessons from Microsoft AI researchers who developed a new deep neural network model that can learn from multiple natural language understanding tasks at once.

Who benefits from the research breakthroughs made in the China-based research labs of American artificial intelligence (AI) companies? Just five years ago, that question hardly ever came up, and if it was asked, the answer often centered on the shared benefits of global research. The field of machine learning has made major strides, China's technology markets and its surveillance apparatus have boomed, and technological competition has moved to the center of the US-China relationship. What do all these changes mean for the overseas research labs of leading American technology companies? To answer that question, it's useful to zoom in on a specific research breakthrough to examine the ideas, institutions, and people involved in it. By tracing those factors over time, a better assessment can be made on where the benefits from this research flow to, and how government policies and corporate practices can best shape those flows.

The conversation around Artificial Intelligence usually revolves around technology-focused topics: machine learning, conversational interfaces, autonomous agents, and other aspects of data science, math, and implementation. However, the history and evolution of AI is also inextricably linked with waves of innovation and research breakthroughs that run headfirst into economic and technology roadblocks. There seems to be an indelible pattern of discovery, innovation, interest, investment, cautious optimism, boundless enthusiasm, realization of limitations, technological roadblocks, withdrawal of interest, and retreat of AI research back to academic settings. These waves of advance and retreat appear to be as consistent as sea waves on the shore. This pattern is vexing to technologists and investors because it doesn't follow the usual technology adoption lifecycle.

Chinese People's Liberation Army (PLA) Air Force officers march past Tiananmen Square in a show of military strength Last month, some of the biggest names in technology signed a pledge promising not to develop lethal autonomous weapons. Coming just after the recent employee-led protest over Google's Project Maven, some have praised these initiatives as ethical and moral victories. For Sandro Gaycken, a senior advisor to Nato, such initiatives are supremely complacent and risk granting authoritarian states an asymmetric advantage. "These naive hippy developers from Silicon Valley don't understand – the CIA should force them," says Gaycken, founder of the digital society institute at ESMT, a Berlin-based business school. Gaycken's hard advice reveals a schism emerging in the future development of AI for military purposes. On the one side are those that believe pursuing the development of military AI will lead to an unstoppable arms race. On the other side, people like Gaycken believe the AI arms-race has already begun.

Artificial intelligence, robots, and automation are rising in importance in many areas. As noted in the recent book, "The Future of Work: Robots, AI, and Automation," there are exciting advances in finance, transportation, national defense, smart cities, and health care, among other areas. Businesses are developing solutions that improve the efficiency and effectiveness of their operations and using these tools to improve the way their firms function. Yet there also are concerns about the impact of these developments on jobs and personal privacy. A Pew Research Center national survey revealed considerable unease about emerging trends.